I. Field of the Invention
This invention relates generally to a computer controlled imaging system, and more particularly to a system for creating realistic images of a three-dimensional scene on a two-dimensional medium wherein multiple objects can be simultaneously undergoing processing in plural channels.
II. Discussion of the Prior Art
Operators of complex vehicular systems, such as tanks, planes or ships, must receive extensive training to become proficient in their operation. It is common practice to rely on various types of system simulators in the training of these operators since it may be prohibitively costly and impractical to do this training on the actual vehicle. This is particularly true when training operators to respond to emergency or hazardous situations. For this training to be effective, it is highly desirable that all of the stimuli received from the training simulator be as close as possible to the actual stimuli provided by the vehicle itself.
The two dimensions of the realism of a visual image are its fidelity and its dynamics. Ideally, the display image should be indistinguishable from the view of a real image and should move smoothly in real time in response to actions taken by the vehicle operator. It is common practice to use various types of computer-generated imagery to provide this display. Such prior art systems are typically very expensive and still fall short of the ideal of high fidelity and smooth real-time dynamics.
This invention relates to a novel processing architecture referred to as "multiple object pipelined processor" to provide highly realistic synthesized imagery at a significantly lower cost and complexity vis-a-viz prior art systems.
A comprehensive discussion of a prior art animation system is contained in a Graf, et al U.S. Pat. No. 4,645,459, entitled COMPUTER GENERATED SYNTHESIZED IMAGERY. That patent is assigned to Honeywell, Inc. who was the assignee of the instant application until its assignment to Hughes Simulation Systems, Inc. effective Jan. 1, 1989. Graf describes "computer generated imagery" (CGI) in which the objects and surfaces for constructing video-displayable scenes are derived from purely mathematical models stored in the form of points, which define the limits of the objects and surfaces. Such mathematical models are readily manipulated by digital computers to efficiently provide excellent surface representation, accurate positioning in the scene, correct illumination and correct perspective. Although CGI is very strong in manipulation of scene objects, it is lacking in realism in that the object itself cannot be realistically presented. This causes the resulting scene to appear cartoonish, particularly in the representation of highly structured objects, such as trees, grass and detailed man-made objects.
Another known approach is so-called "computer synthesized imagery" (CSI). In contrast to the CGI approach where the digital computer constructs the image of the object, in CSI the object images are pre-recorded using a photographic process, or its equivalent, and then assembled into an object database which can be accessed by a digital computer. Very realistic scenes may be generated, providing that the database contains exactly the scene and perspective called for. A major weakness of this approach is that it is not practical for any complex system to store the very large number of camera viewpoints required. Such systems are thus somewhat unrealistic in that, although the objects portrayed may be true images, they are not appropriately sized, rotated and set in the proper perspective for the point of view required at any given instant. The aforereferenced Graf patent discloses an embodiment which combines the image manipulative strengths of CGI with the realistic portrayal of objects of CSI to form computer-generated synthesized imagery (CGSI).
With reference to FIG. 3 of the Graf patent, it depicts an implementation of a vehicle simulation system using a CGSI approach to generate the display. In this arrangement, a field-of-view computer is programmed to coordinate the display of a particular point of view based upon a vehicle attitude and position in the gaming area provided by the vehicle simulation computer. The gaming area database contains the identity and position of all objects in the gaming area, but not the image data of the object itself. All object image data is contained in the object library database which is shown as being replicated for each of the three processing channels. For each instance of every object to appear in the final scene, the field-of-view computer provides the object identity, its position and orientation that it must have in the final scene. The object controllers fetch the object images from the object library database and present them to the object processing channels which perform the required manipulations to fit the objects onto the scene. Identical processing channels are provided to process the surface and special effects associated with a given object.
FIG. 31 of the Graf patent shows a more detailed representation of the object processing channel. The object is received from the object database in analog video form and is converted by the analog-to-digital converter card 30 into a digital format where it is stored in the high-speed X/Y access memory 32. The preferred format in that embodiment has 512 lines, each line having 512 pixels and each pixel comprising an 8-bit word representing 256 intensity levels. The object image transformation is accomplished with a two-pass warp algorithm, a first pass to transform the object pixels of a given column into their correct row positions and a second pass to transform all pixels in a given row to their correct column position.
To accomplish this, image data is read from a memory 32 sequentially, in column-by-column order, and is passed through the look-up table card 34 to provide intensity modification to the image and then presented to the Y access warp card 36. The results of the Y access warp are stored in a high-speed X/Y access memory 38. The image is read from memory 38 in row order and presented to the X access warp card 40 for the second pass warp. The output of the X access warp is presented to the scene construction module of FIG. 3 of the Graf patent. The two pass warp process is shown graphically in FIG. 30 of the Graf patent. It shows a 512.times.512 pixel input imagery frame from the optical disk warped to be positioned on a 512.times.512 pixel video screen.
Given the equal size of the image frame with the resultant video screen, this warp accomplishes the combination of size rotation and position of the object on the video screen. In typical situations, most objects are relatively small compared to the size of the screen and Graf recognizes that it is generally inefficient to warp small objects from an input frame the size of the output scene. He thus suggests as a solution to divide the input frame into a fixed array, e.g., 4.times.4 of smaller input frames, each having independently programmed warp factors. With this approach, up to 16 small objects may be loaded into memory 32 and independently warped to the output screen. However, this solution does not address the problem of properly occluding these objects or translating them to their final preferred screen location.
While the CGSI approach taught by Graf provides a substantial improvement over the prior art in the generation of real-time realistic imagery, it is nevertheless limited in its ability to deal with highly dynamic scenes involving a very large number of objects. These limitations arise due to the maximum rate which objects may be accessed from a single object database and the maximum rate at which objects may be warped, given a single channel warp processor and the processing inefficiency associated with combining sizing, rotation, and positioning in a single warp process.